{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='HOME'></a>\n",
    "# CHAPTER 11 Concurrency and Networks\n",
    "## 開發與網路\n",
    "\n",
    "* [11.1 開發](#Concurrency)\n",
    "* [11.2 網路](#Networks)\n",
    "\n",
    "Time is nature’s way of keeping everything from happening at once.  \n",
    "Space is what preventseverything from happening to me.  \n",
    "— Quotes about Time\n",
    "\n",
    "先前的範例都是在一台電腦一次運行一次，但是可以利用技術做到分散式計算等更強大的功能。\n",
    "\n",
    "好處有\n",
    "* 提升性能，可以不用因為某部分運行較慢而塞住後面的程序\n",
    "* 提升安全性，同時運作多個相同任務防止意外\n",
    "* 把程式拆解得更細小更好維護與理解\n",
    "* 網路傳送與接收數據"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "<a id='Concurrency'></a>\n",
    "## 11.1 開發\n",
    "[回目錄](#HOME)\n",
    "\n",
    "一般而言在計算時速度變慢時多半是因為__I/O___的限制以及__CPU限制__  \n",
    "I/O的速度比起CPU計算上慢上許多，  \n",
    "CPU算得較慢時可能是遇到科學相關需要大量計算或是圖形相關的計算導致變慢\n",
    "\n",
    "程式在開發時一般會有兩種技術，__同步(synchronous)__與__非同步(asynchronous)__\n",
    "\n",
    "* 同步是指說程式在執行時是一個接著一個，A指令做完才做B指令，最後在C指令\n",
    "* 非同步是可以讓A B C三種指令同時在一個程式裡執行"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__佇列(Queue)__中文也翻作隊列，顧名思義是一種像排隊一樣的概念，  \n",
    "以生活中的情況為人們一個接一個的從隊伍後面加入排隊，而窗口的服務人員則從最前面的民眾一個接一個處理事務。(http://emn178.pixnet.net/blog/post/93475832-%E4%BD%87%E5%88%97(queue))\n",
    "\n",
    "本書作者以洗碗流程作為範例，分成兩段工作過程，清洗與烘乾。  \n",
    "假設一個人做事，可以有兩種做法，\n",
    "1. 洗完一個盤子就拿去烘乾\n",
    "2. 先洗完全部的盤子，在統一把盤子烘乾\n",
    "\n",
    "若要改善效率最快的方法就是找幫手，一人負責洗碗一人負責烘乾，但是這樣會遇到一個問題，如果洗盤子的速度大於烘乾盤子時，是要等待烘乾的人員閒置後在遞交盤子給他，還是先行放置在桌上，在繼續洗下一個盤子，等他有空時再自行拿取，前者就是同步的概念，後者則為非同步的概念。\n",
    "\n",
    "假設水槽中的盤子們是佇列(Queue)中的工作項目，可以進行同步與分同步的工作流程。  \n",
    "* 同步：把水槽中的髒盤子給第一個閒置的洗碗人員洗，洗完後等烘乾人員閒置後再把盤子給他\n",
    "* 非同步：洗盤子的人員洗好就將盤子放置在桌上後繼續清洗下一個，烘乾人員閒置時就去看桌上有無盤子可以清洗。\n",
    "\n",
    "![Alt text](http://i.imgur.com/mtFun5p.png \"時間花費表\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 原始洗碗流程\n",
    "\n",
    "```python\n",
    "import os\n",
    "import time\n",
    "from datetime import datetime\n",
    "\n",
    "def washer(dishes, now_):\n",
    "\tfor dish in dishes:\n",
    "\t\tnow = datetime.now()\n",
    "\t\tprint('Washing', dish, ', time:', now - now_, ', pid', os.getpid())\n",
    "\t\ttime.sleep(1)\n",
    "\t\tdryer(dish, now_)\n",
    "\n",
    "def dryer(dish, now_):\n",
    "\tnow = datetime.now()\n",
    "\tprint('Drying ', dish, ', time:', now - now_, ', pid', os.getpid())\n",
    "\ttime.sleep(2)\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "\tnow_ = datetime.now()\n",
    "\tdishes = ['dish-1', 'dish-2', 'dish-3', 'dish-4']\n",
    "\twasher(dishes, now_)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Washing dish-1 , time: 0:00:00 , pid 13204\n",
      "Drying  dish-1 , time: 0:00:01.010387 , pid 13204\n",
      "Washing dish-2 , time: 0:00:03.023734 , pid 13204\n",
      "Drying  dish-2 , time: 0:00:04.027234 , pid 13204\n",
      "Washing dish-3 , time: 0:00:06.027493 , pid 13204\n",
      "Drying  dish-3 , time: 0:00:07.039876 , pid 13204\n",
      "Washing dish-4 , time: 0:00:09.052473 , pid 13204\n",
      "Drying  dish-4 , time: 0:00:10.060635 , pid 13204\n"
     ]
    }
   ],
   "source": [
    "import subprocess\n",
    "ret = subprocess.getoutput('python Data/dishes.py')\n",
    "print(ret) #執行需時間，請等候"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 行程or處理程序(Processes)\n",
    "\n",
    "[wiki 行程](https://zh.wikipedia.org/wiki/%E8%A1%8C%E7%A8%8B)\n",
    "\n",
    "下列範例可以模擬兩個人員在進行分工合作，洗碗為主行程，烘乾則為另開的行程，所以洗碗者不必等到烘乾完畢就可以洗下一個盤子\n",
    "\n",
    "```python\n",
    "import multiprocessing as mp\n",
    "import os\n",
    "import time\n",
    "from datetime import datetime\n",
    "\n",
    "def washer(dishes, output, now_):\n",
    "\tfor dish in dishes:\n",
    "\t\tnow = datetime.now()\n",
    "\t\tprint('Washing', dish, ', time:', now - now_, ', pid', os.getpid())\n",
    "\t\ttime.sleep(1)\n",
    "\t\t#把東西丟給其他行程後繼續執行下一個\n",
    "\t\toutput.put(dish)\n",
    "\n",
    "def dryer(input, now_):\n",
    "\twhile True:\n",
    "\t\tdish = input.get()\n",
    "\t\tnow = datetime.now()\n",
    "\t\tprint('Drying ', dish, ', time:', now - now_, ', pid', os.getpid())\n",
    "\t\ttime.sleep(2)\n",
    "\t\tinput.task_done()\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "\tnow_ = datetime.now()\n",
    "\t#建立佇列\n",
    "\tdish_queue = mp.JoinableQueue()\n",
    "\t#創建行程(烘乾人員)\n",
    "\tdryer_proc = mp.Process(target=dryer, args=(dish_queue, now_,))\n",
    "\tdryer_proc.daemon = True\n",
    "\t#啟動行程(上班囉)\n",
    "\tdryer_proc.start()\n",
    "\t#time.sleep(1)\n",
    "\t\n",
    "\tdishes = ['dish-1', 'dish-2', 'dish-3', 'dish-4']\n",
    "\twasher(dishes, dish_queue, now_)\n",
    "\tdish_queue.join()\n",
    "```\n",
    "p.s. 在ipython中非主程式的行程print不出來，請自行在本機端cmd跑，或是把print改成寫成實體檔案方可看見結果，  \n",
    "Data資料夾中有dishes_process.py檔案可供使用  \n",
    "\n",
    "結果:  \n",
    "Washing dish-1 , time: 0:00:00.037144 , pid 10480  \n",
    "Washing dish-2 , time: 0:00:01.047415 , pid 10480  \n",
    "Drying  dish-1 , time: 0:00:01.047415 , pid 7280  \n",
    "Washing dish-3 , time: 0:00:02.060229 , pid 10480  \n",
    "Drying  dish-2 , time: 0:00:03.047613 , pid 7280  \n",
    "Washing dish-4 , time: 0:00:03.063241 , pid 10480  \n",
    "Drying  dish-3 , time: 0:00:05.047959 , pid 7280  \n",
    "Drying  dish-4 , time: 0:00:07.053659 , pid 7280"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 執行緒(Threads)\n",
    "\n",
    "[wiki 執行緒](https://zh.wikipedia.org/wiki/%E7%BA%BF%E7%A8%8B)\n",
    "\n",
    "下列範例可以模擬一個人員洗，兩個人烘的分工合作，全部都為同一個主行程，但是另開兩個線程來處理烘乾工作\n",
    "\n",
    "```python\n",
    "\n",
    "import threading, queue\n",
    "import os\n",
    "import time\n",
    "from datetime import datetime\n",
    "\n",
    "def washer(dishes, dish_queue, now_):\n",
    "\tfor dish in dishes:\n",
    "\t\tnow = datetime.now()\n",
    "\t\tprint (\"Washing\", dish, now - now_, ', pid', os.getpid(), threading.current_thread())\n",
    "\t\ttime.sleep(1)\n",
    "\t\tdish_queue.put(dish)\n",
    "\n",
    "def dryer(dish_queue, now_):\n",
    "\twhile True:\n",
    "\t\tdish = dish_queue.get()\n",
    "\t\tnow = datetime.now()\n",
    "\t\tprint (\"Drying \", dish, now - now_, ', pid', os.getpid(), threading.current_thread())\n",
    "\t\ttime.sleep(2)\n",
    "\t\tdish_queue.task_done()\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "\tdish_queue = queue.Queue()\n",
    "\tnow_ = datetime.now()\n",
    "    #控制要開幾條執行緒\n",
    "\tfor n in range(2):\n",
    "\t\tdryer_thread = threading.Thread(target=dryer, args=(dish_queue, now_))\n",
    "\t\tdryer_thread.daemon = True\n",
    "\t\tdryer_thread.start()\n",
    "\t\t\n",
    "\tdishes = ['dishe-1', 'dishe-2', 'dishe-3', 'dishe-4']\n",
    "\twasher(dishes, dish_queue, now_)\n",
    "\tdish_queue.join()\n",
    "\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Washing dishe-1 0:00:00.000505 , pid 5376 <_MainThread(MainThread, started 18348)>\n",
      "Washing dishe-2 0:00:01.012149 , pid 5376 <_MainThread(MainThread, started 18348)>\n",
      "Drying  dishe-1 0:00:01.012149 , pid 5376 <Thread(Thread-1, started daemon 11096)>\n",
      "Washing dishe-3 0:00:02.012222 , pid 5376 <_MainThread(MainThread, started 18348)>\n",
      "Drying  dishe-2 0:00:02.012222 , pid 5376 <Thread(Thread-2, started daemon 8768)>\n",
      "Washing dishe-4 0:00:03.022536 , pid 5376 <_MainThread(MainThread, started 18348)>\n",
      "Drying  dishe-3 0:00:03.022536 , pid 5376 <Thread(Thread-1, started daemon 11096)>\n",
      "Drying  dishe-4 0:00:04.032120 , pid 5376 <Thread(Thread-2, started daemon 8768)>\n"
     ]
    }
   ],
   "source": [
    "import subprocess\n",
    "ret = subprocess.getoutput('python Data/dishes_threads.py')\n",
    "print(ret) #執行需時間，請等候"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "總而言之，對於 Python，建議如下：\n",
    "* 使用執行緒來解決 I/O 限制問題；\n",
    "* 使用行程、網絡或者事件（下一節會介紹）來處理 CPU 限制問題。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "書中還介紹許多函示庫可以辦到各種不同的佇列(Queue)功能\n",
    "* gevent\n",
    "* twisted\n",
    "* asyncio\n",
    "* Redis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "<a id='Networks'></a>\n",
    "## 11.2 網路\n",
    "[回目錄](#HOME)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [Root]",
   "language": "python",
   "name": "Python [Root]"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
